Effectiveness of the Kozachenko-Leonenko estimator for generalized entropic forms.
نویسنده
چکیده
In this Brief Report we discuss the effectiveness of the Kozachenko-Leonenko entropy estimator when generalized to cope with entropic forms customarily applied to study systems evincing asymptotic scale invariance and dependence (either of linear or nonlinear kind). We show that when the variables are independently and identically distributed the estimator is only valuable along the whole domain if the data follow the uniform distribution, whereas for other distributions the estimator is only effectual in the limit of the Boltzmann-Gibbs-Shanon entropic form. We also analyze the influence of the dependence (linear and nonlinear) between variables on the accuracy of the estimator between variables. As expected in the last case the estimator loses efficiency for the Boltzmann-Gibbs-Shanon entropic form as well.
منابع مشابه
On the effectiveness of a binless entropy estimator for generalised entropic forms
In this manuscript we discuss the effectiveness of the Kozachenko-Leonenko entropy estimator when generalised to cope with entropic forms customarily applied to study systems evincing asymptotic scale invariance and dependence (either linear or non-linear type). We show that when the variables are independently and identically distributed the estimator is only valuable along the whole domain if...
متن کاملOn the Kozachenko-leonenko Entropy Estimator
We study in details the bias and variance of the entropy estimator proposed by Kozachenko and Leonenko [10] for a large class of densities on Rd. We then use the work of Bickel and Breiman [2] to prove a central limit theorem in dimensions 1 and 2. In higher dimensions, we provide a development of the bias in terms of powers of N−2/d. This allows us to use a Richardson extrapolation to build, i...
متن کاملAnalysis of k-Nearest Neighbor Distances with Application to Entropy Estimation
Estimating entropy and mutual information consistently is important for many machine learning applications. The Kozachenko-Leonenko (KL) estimator (Kozachenko & Leonenko, 1987) is a widely used nonparametric estimator for the entropy of multivariate continuous random variables, as well as the basis of the mutual information estimator of Kraskov et al. (2004), perhaps the most widely used estima...
متن کاملEfficient multivariate entropy estimation via k-nearest neighbour distances
Many statistical procedures, including goodness-of-fit tests and methods for independent component analysis, rely critically on the estimation of the entropy of a distribution. In this paper, we seek entropy estimators that are efficient in the sense of achieving the local asymptotic minimax lower bound. To this end, we initially study a generalisation of the estimator originally proposed by Ko...
متن کاملRisk Estimation and Feature Selection
For classification problems, the risk is often the criterion to be eventually minimised. It can thus naturally be used to assess the quality of feature subsets in feature selection. However, in practice, the probability of error is often unknown and must be estimated. Also, mutual information is often used as a criterion to assess the quality of feature subsets, since it can be seen as an imper...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Physical review. E, Statistical, nonlinear, and soft matter physics
دوره 80 6 Pt 1 شماره
صفحات -
تاریخ انتشار 2009